A prototype system for rule-based expressive modifications of audio recordings

A prototype system is described that aims to modify a musical recording in an expressive way using a set of performance rules controlling tempo, sound level and articulation. The audio signal is aligned with an enhanced score file containing performance rules information. A timefrequency transformation is applied, and the peaks in the spectrogram, representing the harmonics of each tone, are tracked and associated with the corresponding note in the score. New values for tempo, note lengths and sound levels are computed based on rules and user decisions. The spectrogram is modified by adding, subtracting and scaling spectral peaks to change the original tone’s length and sound level. For tempo variations, a time scale modification algorithm is integrated in the time domain re-synthesis process. The prototype is developed in Matlab. An intuitive GUI is provided that allows the user to choose parameters, listen and visualize the audio signals involved and perform the modifications. Experiments have been performed on monophonic and simple polyphonic recordings of classical music for piano and guitar.